Insights from the project

Based on a literature review and semi-structured interviews, the study identified use cases for predictive analytics and critical factors for successful implementation, including policy gaps. Our findings indicated the evidence base for predictive analytics in healthcare was immature. We identified a lack of high quality prospective studies of effect.

As is often the case with rapidly developing technologies, the lack of evidence did not deter healthcare organisations from using predictive analytics. Typically, larger well-resourced healthcare organisations were the adopters, while smaller organisations struggled to keep pace due to insufficient investment capabilities and inadequate data management infrastructure. It should be noted that the majority of organisations we spoke to were some way off from implementing predictive analytics at scale, identifying the need for appropriate expertise and governance structures.

We found multiple use cases for predictive analytics in the literature (1). However, most organisations were using predictive analytics in a limited number of areas: to identify individuals at risk for preventable readmissions, hospital acquired infections, sepsis, clinical deterioration and high healthcare utilisation. Few organisations rigorously measured the impact of these initiatives. Nonetheless, the majority claimed reductions in readmissions and healthcare utilisation amongst high utilisers and improved sepsis outcomes, where these tools have been implemented.

The insights regarding successful implementation were the most interesting aspects of our work and remain relevant for organisations implementing digital tools today (2). We interviewed 34 key stakeholders from healthcare organisations across the United States, federal and state level policymakers, commercial and nonprofit vendors. Our interview subjects highlighted three critical areas for successful implementation of predictive analytic tools (figure 1):

• the predictive tool itself

• involving the right people

• organisational readiness

The predictive tool

There is no shortage of data or data-driven modelling in healthcare. Despite the abundance of predictive models in the literature, most are not implemented in clinical workflows. This can occur for several reasons: Many models address issues of marginal interest to health care practitioners and leaders. Almost everyone we interviewed emphasised the importance of addressing an important issue, preferably a key outcome for patients.

Determing and clearly articulating the «right» problem to address was crucial, both in terms of choosing the best predictive tool for the job, but also for clinician engagement. If the tool did not address an issue of sufficient importance to providers and patients, it was less likely to be used.

Health care providers underscored the need for actionable outputs that contributed to workflow optimisation. Model outputs needed to correspond with a predefined set of evidence-based actions. The most accurate predictive analytic tool can fall by the wayside, if the clinician has no idea what to do with the information. The most successful tools made it easier for users to act appropriately and proactively.

Involving the right people

Healthcare organisations and vendors consistently reiterated the importance of involving the right individuals throughout the entire implementation cycle (development, validation, implementation and evaluation) A multidisciplinary team approach involving clinical, analytical, IT and deployment skills, was a key success factor.

Securing buy-in from leaders at all levels was considered essential for success. Successful organisations took a life-cycle approach to managing and maintaining these tools and committed to long-term funding and investment in these initiatives. However, leadership buy-in from senior management also must be accompanied by the efforts of clinical champions, for successful uptake (3). Access to a cadre of skilled change agents is an essential part of modern healthcare delivery, yet it is often neglected. Recruiting well-respected clinical champions or thought leaders to promote the tool and its utility amongst peers appeared to improve uptake, according to those interviewed.

Organisational readiness

Organisations that successfully implemented predictive analytics tended to treat data as a strategic asset. These organisations invested in data infrastructure, data security and data governance, ensuring quality control and oversight of the introduction and maintenance of predictive tools. Organisations with an established culture of quality improvement typically had an advantage in terms of managing change and measuring its impacts. Having data on the benefits of predictive analytic tools was helpful in improving uptake and securing financing for further scale-up.

Policy reflections

We found that predictive analytics was not being applied at scale in the United States, with Norway lagging even further behind. In addition to organisational constraints mentioned previously, we identified policy-related barriers to uptake. At the time, policymakers were struggling to develop policy and regulations that promoted increased use of data and analytics, without compromising public safety, privacy and acceptability. It was clear, even in 2015, that policy should focus on building a climate of trust around the data and their use. Four key policy principles emerged:

• Patients, providers and the public should be able to trust the quality of the underlying data,

• Predictive tools should be unbiased and accurate,

• Data should be used meaningfully

• Health data should remain secure and not be misused.

These policy goals were considered necessary for acceptance of predictive analytics and other big data initiatives – in the United States and beyond.

We identified concrete policy measures such as partnering with patients and providers to develop robust processes for consent, data collection, linkage and terms of use. More meaningful use of data was seen as a means of boosting public confidence. This would require access to multiple datasets and the ability to link data, necessitating robust information security and accelerated uptake of data standards. The latter was particularly relevant for Norway’s national Electronic Medical Record (EMR) policy in 2016, as implementation of international health information standards was slow.

By 2016 it became increasingly clear to us that we needed policies to protect patients and individuals from potential harms or discrimination as predictive and prescriptive tools increased in sophistication (4). It is fair to say that this is even more of a concern today. We were careful to point out the need for a regulatory balancing act: on the one hand ensuring safety and lack of bias, while on the other hand being wary of stifling innovation. Many of these overarching policy themes were discussed at the Big Data Symposium that David Bates and I organised with the Commonwealth Fund in the autumn of 2016.

Longer-term impacts

Predictive analytics continue to be routinely used across many centers in the United States, particularly in the areas of readmissions and sepsis detection. In Norway, however, the integration of these tools has been uneven. Existing Norwegian national digital health initiatives were put on hold in 2022, after a decade marked by a litany of catastrophically expensive failures. Ironically, the SARS‑CoV‑2 pandemic provided a brief opportunity for agile policymaking and investment in facilitating rapid data linkage from multiple sources to promote meaningful use of data, in the public interest. Whilst the catalytic effect of the pandemic has abated, initiatives such as the Health Data Services have been established to promote access to registry data. It remains unclear to what extent these initiatives promote meaningful use and improve outcomes. It is clear, however, that there remains untapped potential within Norwegian health data. If appropriately harnessed, health data could inform policy, research and practice beyond our borders.

My work was presented to policymakers in the Harvard/Partners Healthcare system, Tufts and in Norway. Our work continues to be cited by other researchers and practitioners in the field of healthcare analytics. As such we may infer that our findings pertaining to success criteria and barriers to implementation remain relevant today.

Beyond the Harkness fellowship

The Harkness Fellowship provided a unique opportunity to take a deep dive into the American healthcare system. While there is much to admire and much to criticise within the U.S. healthcare system, I was most fascinated by the thriving culture of innovation, often absent in Norway. There was no shame in trying to solve big, audacious problems and less of a fear of failure. The culture of entrepreneurialism and the professional attitude to innovation pervaded the healthcare institutions I was lucky to visit in the United States. I missed that upon my return to Norway, to my role as Head of E-health at the South-eastern Regional Health Authority. During my time in the United States, I learned that no idea is too small and that we can create systems that applaud, harvest, and follow innovation to fruition.

It was a year of «magical thinking» with room for creativity, reflection and the courage to think big(ger). For me, it resulted in leaving my leadership role, to pursue a PhD in operational research, demonstrating the value of using health data and mathematical methods to support problem-solving and policymaking in healthcare.

My research focused on the impact of municipal admissions units (MAUs) – a national initiative aimed at reducing hospital admissions. MAUs had faced media criticism for persistently low bed occupancy rates, and it was unclear how central policymakers had determined the numbers of MAU beds required. Much of healthcare policy and decision-making is based on historical demand and projections of population growth. It pays little heed to the variations in demand and the science of queueing – which can be useful in accurately estimating capacity requirements. Our analyses indicated that the supply of MAU beds far exceeded the demand and that MAUs had not reduced the number of hospital admissions (5, 6). The work demonstrated how the use of relatively simple models and analysis could have informed not only the initial policy but also subsequent planning.

In January 2018, while pursuing my PhD, I was asked by Senator Bernie Sanders to participate as an expert at a live-streamed Medicare for All Town Hall meeting, at the Senate in Washington DC. It was an incredible experience, viewed by over a million people and reported on by the Washington Post. After a year of witnessing the injustices of the American healthcare system, I felt a duty to inform the American public that examples of high quality, equitable healthcare existed. Truth be told, while I applauded the Senator’s initiative, I had little faith that the United States was ready for a single payer system and the necessary conversations regarding prioritisation, resource allocation and gatekeeping.

After completing my PhD at the University of Oslo and Akershus University Hospital (Ahus), I moved to the private sector, serving as Chief Medical Officer for a remote patient monitoring scale-up, Dignio. It was thrillingly out of my comfort-zone and I likened the experience to an «MBA-by-doing». My work involved closely collaborating with clinicians in hospitals and municipalities to redesign health care in a more sustainable manner. This experience inspired me to return to clinical work and after spending the majority of my career working at a systems level, I am now retraining as an oncologist. It is a challenging transition but there is something extremely satisfying bringing my experience from public health, leadership and the use of data and technology to a clinical setting. I am particularly interested in bringing the Common Sense Oncology movement to Norway. Common Sense Oncology promotes cancer care and research focusing on improving outcomes important to patients and their families, such as overall survival and quality of life (7-8).

I am most grateful for relationships established during the Fellowship, both within the Commonwealth Fund and the Harkness network. This unique asset continues to be a source of advice, friendship and new opportunities. My mentor, David Bates, remains a trusted advisor and friend. I had the privilege of being temporarily seconded to the data-analytics team at the Health Foundation (a leading UK health policy think-tank), during my PhD. This was via the UK Harkness Fellow Adam Steventon. I also recruited Luke O’Shea, another 2015–2016 fellow, to the Dignio Advisory Board, as we looked to expand further in the UK. These are just some examples among many.

Finally, it would be remiss to discuss the impact of my Harkness experience without mentioning my family. It was a magical year for us all: a new house, a new school (Quaker, no less), new colleagues and new friends. Moving to a different country is not without its challenges but it can be both life-changing and life-affirming.